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During the past two decades, research designs of outcome studies in 
educational psychology have increasingly combined' what Cronbach (1957) 
called "the two disciplines of scientific psychology". Correlational 
research attentive to individual differences became more frequently incor- 
porated into experimental designs that sought to control and manipulate 
types of treatments. Nearly a thousand such studies were recently reviewed 
N ijy Cronbach and Snow '(1977) with results summarized as modest if not 
equivocal. Recommendations for future research have included the use of 
multivariate analysis of several dependent variables (Cronbach and Snow, 
'1977), the use of more complex designs that account for interactions 
between a greater number of .^dependent variables (Jenkins, 1 977), and 
more extensive theory development and clarification of these interactions 
as a guide to more fruitful research (Blommers, 1970; Cronbach and Snow, 1977) 
' While debate continues on types of teaching methods most facilitative I 
• of^different outcomes for different types of students, regular evaluation 
of instruction continues within college settings. Student evaluation of ~ 
instruction forms are used in many college and university settings to 

\ t 

* provide vfeedjack to instructors on their teaching strengths and weaknesses, 
as t well as to provide information to adminsitfator.s making promotion^and 
salary decisions. Although considerable psychometric rigour has been 

put into the construction and validation of the items ^n these"Vating 

1 

forms, the item pools have not tended to focus upon possible attribute-* 

* # 

treatment interactions that might effect how students view their instruc- 

r . . * — - \ 

tors-, „ Typically, some attempts. Jpfve been made To account for achievement 

" ~( n what is your G.P.A. ?"P and motivation ( M was this course, required for 

t \ L * ^ 

V your. major?"), but mor£ subtle personality andJCogaiii-yle: style variables 

barely seeim to be incorporated into existing item pools according to 

9 
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the available literature. 

Some experts in cognitive style research havfe strongly recommended 
Studying the interaction -of cognitive styles and student ratings of • 
instructors (Messick, -1 970; Witkin, 1*976). Initial studies using tra- 
ditional measures <ofV cognitive styles have reported some significant 
interactions between these styles and student perceptions of teaching 
(Crockett," 1 975;. Witkin, 1 976; Wright and ' Richardson , 1977,- fecker and 
Bain, 1978). This suggests potentially fruitful use of cognitive style 
and cognitive development items in evaluation of . instruction question- 
naires. However, there are several major difficulties with pursuing ' 
this task: 1) the psychometric rigour of some of these measures has 
been brought into .question, 2) most cognitive. style and cognitive develop- 
mental measures .are too cumbersome for routine mass administration and 
scoring, and not easily .transformable into a few select items, and 
3) cognitive.,style and. devel opmental theories and measures .are diverse, 
often loosely defined, and unclear inthei> 'interrelationships' with 
eachothes. ' The purpose of thia study -is to take some initial steps towards 
reducing these difficulties. , ■ . ^ 

- • • • Measu/es* 

Cognitive style and cognitive developmental measures were Selected on 
therbasis of wide usage, research respectability relative to',other measures 
and diversity of metbocL. For these reason*" it was decided to measure field 
independence wfth the Group Embedded Figures 'Test, .cognitive complexity * 
with Bieri's nodi f fed versio> of -KeTly's. Rep Grid Inventory, and .Integra-/ 

tive complexity with the Paragraph. Completion "Test. 

-* ' . *.'*** t . 

Field dependency- independence/ 1s a bipolar cognitive style defined as' 

* * i % * ' ~ * 

"the extent to which a person is able to deal- with a part qf a field- 
separately from the fierlcj as a whole,,' or the. exteht/to whi.ch*he/stfe is 
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able to disembed items from an organized context 5 ". . . the extent to 
which he is analytical . " (Witkin, 1976, pp. 41-42). A review of 
accumulated educatiorf-rel a4;ed findings across numerous studies (Witkin, 
Moore, Goodenough an (J Cox; 1 977) suggests that field independent stu-. 
dents, in comparison to field dependent students, are 1) less reliant on 
external references and therefore less likely' to require externa-lly 

, defined goals and reinforcements, 2) v les* attentive to and therefore 
less able to remember social material, 3) more likely to make use of 
mediational processes such as analyzing, structuring, abstracting, a*nd 
general principles, and 4). less likely to have difficulty in accepting 
the irrelevance of salient attributes in concept .learning. The GEFT 
used in this study to measure field independence is a timed test consisting 
of 18 complex designs from each of which the subject musi disembed a 
specified geometric figure; lower scores indicate greater field dependence. 

Cognitive complexity is also a bipolar cognitive style, which has 
its roots in Kelly's (1955) theory' % that .the master cognitive motive is 
the tendency of the organism to* move in the direction of better predic- 
tion of others' behavior. Kelly's Role Construct Repertory Test (Rep 
Test) for eliciting and analyzing an individual's personal constructs 

. for social judgment was later modified into, a 10 x 10 grid form With ^ 
constructs already provided (Bieri et al . , 1966). The subject rates 
ten different roles (i.e.' self, parent, friend, -etc.) on each of the 

'ten constructs using a sfx-point tickert-type ratirf^ scale. W^thin'each 
role, all possible pairwise'compa'risons of ratings are made (45), with 
a score of 1 given, for each comparison in which ratings differed and a 

c 

score of 0 for each comparison in which ratings were the same. In thrs 
way, 450 pairwise comparisons are made for each subject. The concept of 
cognitive complexity based upon this modified Rep Test was explicated * 
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' by.Bieri, et al. (1966) as an information processing variable that helps 
predict how an individual transforms specified behavioral information into' 
social or clinical judgments.,. It reflects the relative degree of differ- 
entiation of a person's system for' construing behav.ior. Cognitive complexity 
* defined as the"capacity *to construe social behavior in a multidimensional 
way. Scoring methods are described in the n&xt section-. ' 

In contrast to the ^lStively stable and bipolar characteristics of' ' • 
. field -independence and cognitive complexity, integrative complexity is' ' 
based upon a developmental stage theory. The stages are hierarchically ' 
integrated, form an invariant succession'^ individu^ development, 
invol.ve qualitative differences in modes of thinkingTetween stages and ' 
'an underlying holistic organization of thought at each stage (WithereVl, 1 
•1978). One way in which development is conceptualized is as a movement 
from a concrete to an increasingly abstract conceptual system. According - 
to this theory, more concreteness represents minimal differentiation and 
.little or no integration of concepts, while the opposite is- true of nine ' 
■abstrac.tness. The greater an individual's, abstractness , the greater! ' > 
.his or her ability to consider' alternatives , to transcend immediacy ^ 
to relate" facets'- of the world in terms pf their interrelatedness (Harv.ey^ 
Hunfand .Schroder, 1961). % The .PCT used to measure integrative complexity 
involves elaborating six sentence stems-into paragraphs (i.e. "When rlam 
criticised . . .'», "When I*am told,what to do ... .»). '* 

In' addition to these three measures, two items were constructed /for • 

r 

use™ studenyeval'uation of instruction questionnaires that, began .'wjth • 
■the -sentence (fern: "I lea'rn best when the task is to . . .» and provided 
vaHous alternatives for the students to rank order. One item R roVfkd » 
fivevanernativeif/shjoned after Bloom's (1956) taxonomy W the cog- 
nitiv r e dqmaifl including knowlejjt of facts, understanding of basic' frin- 



^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 
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ciples, application, analysis and synthesis. The other item provided 
three alternatives .fashioned after Perry's (1968) developmental theory in- 
cluding the stage? of dualism, multiplicity and. commitment . These two 
items (see Table"J .for description) were included within an evaluation of 
instruction questionnaire developed from studies by Doyle & Wftitely (1974), 

The above-mentioned. theories share in iommon the concept of a* more 
highly differentiated sense of self on one end of the continuum. Field 
dependence-independence theory does not seem to address m^ltidimensionality 
of social judgment as distinctly as does cognitive complexity, and neither 
of these two cognitive styles includes the ability to integrate concepts 
as does integrative complexity. Both fielti independence and high integrati 

* 7 k 

complexity include the ability to analyze and to abstract. .These differ- 

ences and overlaps between theories make for interrelationships that are 

* * 
difficult to interpret. The rather sparse empirical research along these 

lines, using measures similar to those used in this study, indicate 

cognitive complexity to be negatively correlated with integrative com- ' 4 

plexity (Epting and Wil kins , 1974), and ^field-dependence-independence* to" 

be nonsignificant correlated* with integrative complexity (Stewin, 1976) t 

and with cognitive complexity (Elliott, 1961). * , * 

Because the GEFT, PCT, Rep' test and cognitive develppm^ntal items 

involve very different types of tasks (drawing geometric figures, writing 

paragraphs, rating people,* and ranking preferences), we would expect lit/ 

tie or no method variance. 'And since these measures are based on soffjevyhat 

* . • ' * \ 

differing theories as well, we would expect intercorreUtions J?etw£.6n them 

to be low* However, since the cognitive developmental items and-the P£T 

'aVe each based upon developmental scHemass <toe- would expect .sfcjnj fi cant 



correlations between them. 
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. Method and Procedures 

Subjects for this study were 497 male and female lower division \ 
college and university students from 18 classes in the physical sciences, 
social sciences and humanities. The measures were administered during thnee 
different sessions during the academic quarter. 
-vGEFT • 4 

Subjects were instructed to mark an "x" on the page of the GEFT 

where they were working when- four minutes were announced as passed on 

each section, and to stop drawing when five minutes was announced as 

passed. This. allowed us to investigate the possibly more discriminating 

time limit of four minutes per section for ege students. Since corre- 

lations between four- and' five-minute scores were all in the high .90s, 

the traditional five-minute scores were used in the inter-test analyses. 

PCT - 

• 4 ) 

The PCT was scored by the Ontario Institute for Studies in Education. 
Interrater reliability for the "first forty PCTs was in the high .80s. 
Summary scopes of integrative complexity based upon the PCT were derived 
from averaging the t6p three paragraph scores as recommended by Hunt, et 
aj/ (1978). This score' was then compared with an average of all six para- 
graph scores, yielding a correlation of .95. This confirmed the ade'quacy of 
the traditional .scoring method which was then used for the inter-test 
ajnaly^es. • * 

Rep Test * M 

The Rep Test was s^ped by the traditional method described above of 

Bjeri's "d", with higher scores indicating cjVeater cognitive complexity. 

f . • • ' • 

Because the psychometric rigour of this seoring method seemed .questionable 
to us, also ^scored the Rep Test by a variety of other methods (Trabln, T. 
Doyle, K. , and Wood, P., Note i) from which two were selected as alternatlvje 
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measures of cognitive complexity. The first alternative measure is the" ' 
average of^ the standard deviations f<(£ each column (the ratings for each role 
on ten constructs); this measure gives the added information of degree of 
difference in ratings while Bieri ,% s ;"d M \\ V £s only the number of different 
ratings. The second alternative measure used is the first eiqen value 
derived from a- factor^analysis of the rows (ratings for each construct' 
across ten roles) ipr each -individual . The greater the first e\gen. value, 
the more uoidimensvonal (or cognitively simple) a subject's social judgment; 
>this nlethod would seem to most accurately fit the definiti.on of cognitive 
complexity as multidimensionality of social judgment. A detailed discussion 
of the statistical properties within and interrelationships' between these 
and other Modified Rep Test Grid scoring procedures may be fopd in Trabin, 
T., Doyle, K. and Wood, -P (Note 2). ' 
Cognitive Developmental Items *\ 

Subjects were asked to rank theV favored way of learning as'"] 11 , the 
next best as "2", etc. oh the five alternatives in the Bloom-based item and 
the three alternatives Jn*the Perry-based item. # The, alternatives were then 

weighted with those higher in the developmental hierarchy -given lower weights. 

* * 
I/n this way, if ranking is multiplied by weighting for each alternative 

and then added together , subjects with the highest scores will be those 

whose learning preferences reqi/ire the highest cognitive development, this 

scoring procedure also yields a broader range of scores suitable for correla- 

,tional analyses (see table - 2). 

Inter-Test Analyses ; • 

* * . 

• * 

Scores for the GEFT, PCT, three Rep T^st measures, and two evaluation 
questionnaire items on cognitive development were analyzed for systematic 
relationships within a correlation matrix* Also included in the matrix were - 
data on subjects 1 .sex, GPA and standard deviation of'ratings on twelve* 
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instructpr evaluation itefhs (see table 3).- ' 

Results • 

Results indicate no systematic relationships between any- of,' the cog- 
nitive style or cognitive developmental" measures and sex or GPA with the 
exception of a significant .positive correlation between the PCT and >GPA. 
The PCT is the only measure in this study requiring verbal -facility, and 
this facility is probably an important contributing factor to* the .249 ' 
(p-C.OOO) correlation between GPA and ^T. ' 

Both cognitive developmental items showef mild correlations of .15 
(P<.003) and .14 (p<.006) with the standard deviation of instructor > 
ratings; this suggests a relationship between cognitive development and 
differentiafedness of student evaluations of their instructor across 
different, teaching dimensions. Two of the Rep Test measures (Bieri's "d» 
and the average standard deviation of ratings for ea\h role) bothshowed 
statistically significant correlations of (p<.000) wijh differen- 
tiatedness of student evaluation of instructor ratings. ' these results 
replicate Wright and Richardson's (1977) findings, a'nd recalls their 
argument that cognitive complexity 'is' an important response style to 
be- further studied, The (&FT obtained a negative correlation of -.18 
(p <.002) with differentfatedness of evaluation ratings-, suggesting 
that the more field dependent subjects may be more sensitized to and . 
therefore more able to differentiate between differing instructor, be- 
haviors. This is congruent with the theory that field dependent persons 
are more attentive to and able to remember social- stimuli than are field/ 
independent.. persons (Witkin, 1976). 

Analyses of relationships- between different measures ^cognitive 



*Page 9 



style and cognitive development indicated no systematic relationships 
between cognitive style and cognitive developmental measures , which supports 
Loevyiger's (1966),assertion that developmental stage and bipolar trait ' 
concepts have .fundamental differences that prevent easy .comparison. The 
two cognitive style measures (GE FT and Rep Test) were not significantly 
correlated with eachother, which may be because they measure Afferent . 
traits tfr because they involve different tasks..' 

In {ontrasf-to the above-mentioned paucity of i'nter-test relationships, 
the three cognitive developmental measures (PCT and two cognitive develop- 
mental items) were significantly correlated^ th .one another Respite 'the • 
different tasks they involved. The two items correlated .35 with eachother 
(P<.000>. and .16 with PQT (p < .008).' This clearly .indicates stronger 
relationships of cognitive developmental measures with eachoth'er' than with 
cognitive -style measures. The results, also suggest that the items may pro T 
vid* simpler measures'of developmental constructs similar to those' measured 
by the PCT. . x - , \ 

Concisions ' 
• This study explored the potential incorporation of cognitive develop- 
mental concepts into student evaluation' of teaching questionnaires by inves- 
tigating relationships obtained between representative developmental "items 
and other measures within. a nomological net of cognitive structure. Ini- 

r 

tially, alternative methods for scorirfg the GEFT, PCT and Rep Test were / " 
explored. It appears that standard- deviation and factor analytic scoring 
methods for the Rep Test provide important scoring alternatives to Bieri's 

Cognitive style and cognitive'developmental measures were unrelated to 
sex and GPA with the exception of the PCT which was positively relate* to 
GPA. The verbal facility involved fn-the IpCT tasks may. explain thj|... •.. . 
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, relationship. - . 

\ 

Cognitive development measured by coM±m£ted itmes and cognitive > 
-complexity measured by two Rep Te'st scaring, procedures we)e 'positively 
' related to differentiated of student ratings of their instructor as was 
• field dependence (measured- by lower scores on the GEFT). These results 
.support previous studies' contentions that cognitive' compl exity 'is a' 
response style pertaining similarly to Rep Test and student evaluation of 
instruction. tasks, and that field dependent'subjects are more sensitive 
to and able to differentiate between differing social stimuli. The results 
also ^suggest the potential usefulness and applicability of cognitive'" ' 
developmental items in student evaluation of instruction questionnaires. 

Inter-test analyses indicated no significan^interrelationships between 
measures except between all three cognitive developmental measures., with 0 
each other. This is one approach^ validating and clarifying the ■meaning*' 
of these items. An additional approach has .been «xpl icated in Church, A.T., 
Doyle, K. , and Trabin, T. .(Note 1). Many orfnTVoderate chelations ' .* 
obtained in this, study 'were of high statistical significance 'only because • 
of the large number of subjects and actually accounted for about four- per - 
•cent of the variance. This is not unconfnon tn personal ity research, espe- 
cially within the area of cognitive structures. The constructs being assessed 
by the different measures used in this study, and especially .the relation-" 
ships between them, need = niore definitive explication on both theoretical' 
and empirical grounds in the literature. .The results. of thVs study offer 
some .support to the theoretical differences between stage* theories of ctffe- ■ 
nitive development and bipolar trait cbncepts of cognitjv» style The- ' 
results also suggest that cognitive developmental items cat? >e constructed ■ . 
that are effective and meaningful measures which can be' easily "inCdrpor-- 
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a ted xnto 'Student evaluation* of instruction quest^onnair^. J * 
It is recommended that additional items be developed and 
further ; refined, and that their nomctlogical net be further ela- 
borated with experiments of their* influence" on student evalua- 

5. ' - - * 

tion of teaching • 
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. Table 1 

/: J^gnitive Developmental Items 

1. I learn best wben the task is to (pleas* rani these answers -from 1-5 
with l=be'st, 2=next best, etc.): ♦ ♦ . 

show'a^ idea can be applied to an actual situation 

mastery set of Concrete facts or a body^of information 
closely examine "abstract ideas*and theories 

understand basic principles about a subject 

' p U t toge'the/'sev.eral differing ideas into a theory that makes 

s«nse ,to me 



2.1 learn. best when the task is to (please rank these answers from 1-3 

with l=best* 2=next-.best, etc.): 

t . . - ■ 

take a position 'after considering many ways of interpreting a dif- 

ficult problem . , 

figure out the one right answer to a straightforward problem 
' 31 choose* the best- answer from many possible ones which seem equally 
good ) 



' 

( c J Copyright by University of Minnesota, 1979 
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Table 2 

Descriptive Statistics for* the Measures 



" — r Y 

Measures 


N 


•Mean 


* 

S.D. 


- 

■ 'Minimum 


Maximum 


Item .<rl. ■ 






6 89 


15.00 


71,00 


Ittfn #2 • 




> 

VI fid 


1 75 


6.00 


20,00 


PCP 


370 


• ✓ 
it 77 

lb;// 




8.00 


30.00 


geft . 


347 


12.80 


4tW 


U . UU 


/ 1 r no 


Rep\"d" 


315 


301.44 


36.33' 


166.00 


•367.00 


Rep s.d. 


315 


1.03 


.24 


.39 


1.88. 


Rep'*' factor 


315 


3.85 


.73 


• 1.91 


7.20 


Rating s.d. 


389 


.76 


. .25 


O.OO 


1.91 


GPA* 


249 


Z.53 


.95 


1.00 


5.00 


Serf 


268 


-1.47 


.50 


1.00 


2.00 



* GPA of 3.6-4.0 was" coded "1% with lower GPA. groupi ngs coded with^ 
numbers, increasing to 5.. 
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Table 3 

ntercorrelation Matrix for Measures t 











Rep Rep Rep Rating ' 


Measures 


Item #1 


Item n 


PCT 


GtFT "d" s.d. Factor, s.d. GPA Sex 


- ■ « t • ■■ 


hem n 


-- 








Item r2 


.35*** 








PCT 


.16* 


.16* • 




V 

* 


GF FT 


-.05 


-.07 


-.06 




Rep "d" 


.04. 


-.06 


-.04 


-•.01 - . , 


Rep s.d. 


.09 


.04 


-.07 


- - .03 .57***--' 


Rep factor ' 


-.06 * 


.02 


.00 


.06 ^18**. 27*** -- 


t 

Rating s.d. 


-.15** 


.14* 


-.08 


-.18**. 22**. 22** .04 


GPA**** 


.06 


.0J9 


-.25 


-.03 .04 .03 -.02 -.02 


Sex 


.07' 


.04 


.00 


.15 .11 .08 , .05 -.13 .13 — 


• 

t Pairwise deletion results 


in -N' 


s for each correlation of^between 186 and 3 


* P<,.01 








** p<.005 










*** p<;.001 











**** Because of the GPA coding described in Table 2, the above correlations show 
reversed signs from what would obtain if actual rather than coded GPAs had 
been.us^d. » , ■ " ~ 



